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REAL PARTY IN INTEREST 

This application has been assigned of record to Sony Pictures Digital, Inc., 
having a place of business at 10202 W. Washington Blvd., Culver City, CA 90232. The 
assignment was recorded in the records of the United States Patent and Trademark Office at 
Reel/Frame 014441/0734 on September 2, 2003. 

RELATED APPEALS AND INTERFERENCES 

There are no related appeals or interferences. 

STATUS OF CLAIMS 

This is an appeal from the final Office Action dated November 9, 2004, finally 
rejecting Claims 1-26. Claims 1-26 are therefore on appeal. 

STATUS OF AMENDMENTS 

No claims have been amended in the present application subsequent to the 
receipt of the final Office Action dated November 9, 2004. 

SUMMARY OF INVENTION 

The present application relates to computerized signal processing methods and 
systems. The techniques described adjust the tempo of an audio recording to match audio 
events to events in an audio-visual recording. In an exemplary embodiment described in the 
present application, a reference or marker is received that indicates a location in a recorded 
signal where an audio segment is desired. A tempo is provided for the audio segment such 
that the audio segment fits within the section defined by the reference or marker. (See 
present application, page 3, paragraph [0008] - [0009].) 

For purposes of illustration, Figure 1 in the present application includes a 
video file 122 with video frames depicted graphically on a track in track section 120. Audio 
files 124, 126, and 128 are also represented graphically on tracks in track section 120. A time 
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marker 170 can be provided to indicate a desired location in the audio or video in which an 
particular audio segment should fit. If time marker 170 marks a video frame that occurs 
7.007 seconds into the video from the start, audio for that section is adjusted to have a tempo 
that fits the audio section into the marked section of video. For instance, audio file 124 in 
Figure 1 is adjusted to the section of video file 122 marked by time marker 170. Figure 2 
shows the audio file 124 after the tempo has been adjusted to fit the audio file 124 within the 
segment of video file 122 marked by time market 170. One result from the tempo adjustment 
is to match audio to video. (See present application, page 9, paragraphs [0031] - [0032]; see 
also Fig. 1 .) Accordingly, a person can coordinate audio or music with specific sections of 
video. 

ISSUES 

In the Office Action dated November 9, 2004, Claims 1-26 were rejected as 
unpatentably obvious under 35 U.S.C. § 103(a) over U.S. Patent No. 5,642,171 (Baumgartner 
et aU in view of U.S. Patent No. 6,188,396 fBoezeman et aU . The overall issue on appeal is 
whether the Office Action has established a prima facie case of obviousness. The underlying 
issues are whether there is any suggestion or motivation to combine the Baumgartner et al. 
and Boezeman et al. references and whether the Baumgartner et al. and Boezeman et al. 
references, alone or in combination, teach or suggest all of the limitations of Claims 1-26. 
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GROUPING OF CLAIMS 

The grouping of the claims is as follows: 

Claims 1, 10, 15, and 21 are independent. Claims 2-9 are dependent on Claim 
1, adding further features to Claim 1. Claims 1 1-14 are dependent on Claim 10, adding 
further features to Claim 10. Claims 16-20 are dependent on Claim 15, adding further 
features to Claim 15. Claims 22-26 are dependent on Claim 21, adding further features to 
Claim 21. 

Claims 1-9 are grouped together as being directed to a method of matching 
audio events to other audio or video. 

Claims 10-20 are grouped together as being directed to a system for adjusting 
tempo of an audio segment to fit the audio segment to a marked section in a recorded signal. 

Claims 21-26 are grouped together as being directed to a graphical user 
interface configured to provide for the creation of an audio visual production. 

To the extent that the claims in these groups are argued separately below, the 
claims do not stand or fall together. 
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ARGUMENT 

I. LEGAL STANDARD FOR OBVIOUSNESS 

In the present application, Claims 1-26 have been rejected under 35 U.S.C. § 

103(a), which states: 

A patent may not be obtained though the invention is not 
identically disclosed or described as set forth in section 1 02 of 
this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as 
a whole would have been obvious at the time the invention was 
made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by 
the manner in which the invention was made. 

The legal standards under 35 U.S.C. § 103(a) are well-settled. Obviousness 
under 35 U.S.C. § 103(a) involves four factual inquiries: (1) the scope and content of the 
prior art; (2) the differences between the claims and the prior art; (3) the level of ordinary 
skill in the pertinent art; and (4) secondary considerations, if any, of nonobviousness. See 
Graham v. John Deere Co. . 383 U.S. 1, 148 U.S.P.Q. 459 (1966). 

In proceedings before the Patent and Trademark Office, the Examiner bears 
the burden of establishing a prima facie case of obviousness based upon the prior art. In re 
Piasecld, 745 F.2d 1468, 1471-72, 223 U.S.P.Q. 785, 787-88 (Fed. Cir. 1984). "'[The 
Examiner] can satisfy this burden only by showing some objective teaching in the prior art or 
that knowledge generally available to one of ordinary skill in the art would lead that 
individual to combine the relevant teachings of the references."' In re Fritch , 972 F.2d 1260, 
1265, 23 U.S.P.Q.2d 1780, 1783 (Fed. Cir. 1992). 

As noted by the Federal Circuit, the "factual inquiry whether to combine 
references must be thorough and searching." McGinlev v. Franklin Sports, Inc. , 262 F.3d 
1339, 60 USPQ.2d 1001 (Fed. Cir. 2001). Further, it "must be based on objective evidence 
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of record." In re Lee . 277 F.3d 1338, 61 USPQ.2d 1430 (Fed. Cir. 2002). The teaching or 
suggestion to make the claimed combination must be found in the prior art, and not in the 
applicant's disclosure. In re Vaeck . 947 F.2d 488, 20 USPQ.2d 1438 (Fed. Cir. 1991). The 
mere fact that references can be combined or modified does not render the resultant 
combination obvious unless the prior art also suggests the desirability of the combination. In 
re Mills . 916 F.2d 680, 16 USPQ.2d 1430 (Fed. Cir. 1990). "It is improper, in determining 
whether a person of ordinary skill would have been led to this combination of references, 
simply to '[use] that which the inventor taught against its teacher.'" Lee (citing W.L. Gore v. 
Garlock. Inc. , 721 F.2d 1540, 1553, 220 USPQ 303, 312-13 (Fed. Cir. 1983)). 

II. REJECTION OF CLAIMS 1-26 UNDER 35 U.S.C. § 103(a) 

In Section 4 of the Office Action dated November 9, 2004, the Examiner 
rejected Claims 1-26 under 35 U.S.C. § 103(a) as being unpatentable over U.S. Patent No. 
5,642,171 to Baumgartner et al . in view of U.S. Patent No. 6,188,396 to Boezeman et al. 
This rejection is the only rejection made by the Examiner. Further, Baumgartner et al. and 
Boezeman et al. are the only references cited by the Examiner to support the rejection. For 
the reasons given below, the Appellants submit that the Examiner's rejection of Claims 1-26 
is improper and should be reversed. 

A. The Examiner's Rejection of Claims 1-20 Should be Reversed Because 
Baumgartner et al. and Boezeman et al. do not Combine to Teach or Suggest All 
of the Claim Limitations 

An obviousness rejection cannot be properly maintained where the references used in 
the rejection do not disclose all of the recited claim elements. Independent claims 1,10, and 
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15 all require "tempo being adjusted to fit the audio recording to a section of the recorded 

signal." Specifically, Claim 1 recites: 

providing a tempo for an audio recording to be at least partially 
included in the recorded signal, the tempo being adjusted to fit 
the audio recording to a section of the recorded signal marked 
by the reference. 

Claim 10 recites: 

means for providing a tempo for an audio recording segment to 
be included in the recorded signal, the tempo being adjusted to 
fit the audio recording segment to a section of the recorded 
signal marked by the reference. 

Claim 15 recites: 

provide a tempo for an audio recording segment to be included 
in the recorded signal, the tempo being adjusted to fit the audio 
recording segment to a section of the recorded signal marked 
by the reference. 

Neither Baumgartner et al. nor Boezeman et al. disclose, suggest, or teach "the tempo 
being adjusted to fit the audio recording to a section of the recorded signal marked by the 
reference." The Examiner disagrees. In the Office Action, the Examiner argues: 

. . . since while Baumgartner discloses the limitation of the 
tempo being adjusted to fit the audio recording as the technique 
of the method adjusts the audio tempo to maintain 
synchronization (see col. 6, lines 60-61) or of the method of 
maintains synchronization by adjusting audio tempo (see col. 
12, lines 40-41). Baumgartner lacks of providing a tempo for 
an audio to be at least partially included in the recording signal. 

Boezeman, on the other hand, discloses the limitation of 
providing a tempo for an audio to be at least partially included 
in the recording signal, the tempo to fit the audio recording 
section of the recorded signal marked by the reference as the 
technique of a Sequence Editor which during the course of the 
animation, a pieces of audio is also played. At the simulation 
ending of the animation and audio, a video plays (see col. 6, 
lines 58-62 and see Fig. 3-17). Specially, the synchronization 
of the audio and the animation via the co-occur tool in an 
implementation of specifying both the starting and ending time 
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of the audio with respect to the animation. The synchronization 
of the video and animation via the meet tool is an 
implementation of specifying the starting of the video with 
respect to the ending time of the animation (see Boezeman's 
col. 8, lines 34-40). Thus by using Boezeman's graphically 
Sequence Editor as a reference mark; Audio, Video, 
Animation, Events, and Images can be synchronized with 
respect to starting time and ending time which allows the 
developer a great deal of flexibility. The sequence editor, in 
turn, would provide for the developer a view of the 
multimedia events from the perspective of its time structure 
as opposed to available data flow or user interface 
perspective. 

(Office Action, pages 19-20, boldface in original, underlining added.) 

As such, it appears that the Examiner is relying on col. 8, lines 34-40 in Boezemanet 

al. and the description of the "co-occur" tool. The "co-occur" tool, according to Boezemanet 

al., "causes two parts to automatically start and stop at the same time." ( Boezeman et al. . Col. 

2, lines 54-55.) However, the "co-occur" to.ol described in Boezeman et al. performs a 

completely different operation than that recited in Claims 1,10, and 15. Col. 7, lines 55-65 

of Boezeman et al. states : 

As was stated in the real world example above, the animation 
and audio were to start and stop at the same time. By utilizing 
the present invention, the developer may simply ensure that this 
occurs. Referring to FIG. 10, by selecting first the animation 
player play area 131 and then the audio player play area 143 
and then the co-occur tool 70, it is ensured that the animation 
player part and the audio player part will start and stop 
simultaneously . Since the audio was only about 30 seconds in 
length, the audio will obviously have to be replayed to reach 
the 100 second length of the animation. 

(emphasis added.) Thus, Boezeman et al. does not make any adjustments to the tempo of the 
audio to have it fit within a section of animation, rather the audio is "replayed" so that the 
time duration of the audio matches that of the animation. While the duration of audio (i.e., 
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the number of times the audio is played) may be adjusted by the system described in 
Boezeman et al. , there is no discussion or suggestion of adjusting the tempo of the audio. 

In sum, the Examiner admits that "Baumgartner lacks of providing a tempo for an 
audio to be at least partially included in the recording signal." (Office Action, page 19.) The 
only other reference cited by the Examiner in support of the rejection, Boezeman et al. , also 
fails to disclose or suggest "tempo being adjusted to fit the audio recording to a section of the 
recorded signal," as required by Claim 1. As indicated above, an obviousness rejection 
cannot be properly maintained where the references used in the rejection do not disclose all 
of the recited claim elements. Claims 2-9 depend from Claim 1, Claims 11-14 depend from 
Claim 10, and Claims 16-20 depend from Claim 15. Claims 1-20, thus, all require "tempo 
being adjusted to fit the audio recording to a section of the recorded signal." Accordingly, 
Applicants respectfully request withdrawal of the rejection and allowance of Claims 1-20. 

B. The Examiner's Rejection of Claims 21-26 Should be Reversed Because 
Baumgartner et al. and Boezeman et al. do not Combine to Teach or Suggest All 
of the Claim Limitations 

Claims 21-26 also require claim limitations not disclosed or suggested by the 

combination of Baumgartner et al. and Boezeman et al. . Independent claim 21 requires "the 

reference marker being used to adjust the tempo of at least a portion of the first audio 

recording." Specifically, Claim 21 recites: 

a reference marker which is configured to be selectively located 
by a user, the reference marker being used to adjust the tempo 
of at least a portion of the first audio recording, the tempo 
adjustment being provided to fit the first audio recording to 
a section of the second audio or video recording . 

As discussed in Section A above with reference to Claims 1-20, neither Baumgartner 
et ah nor Boezeman et al. disclose, suggest, or teach adjusting the tempo of at least a portion 
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of an audio recording to "fit the first audio recording to a section of the second audio or video 
recording." In the Office Action, the Examiner points to Figure 10 of Boezeman et al. as 
showing the fitting of a first audio recording to a section of a second audio or video 
recording. (See Office Action, page 5.) However, Figure 10 illustrates the operation of the 
"co-occur" tool, which — as explained above — does not "fit" the audio by adjusting the 
tempo. The "co-occur" tool fits the audio to a section by replaying the audio as many times 
as is necessary to match the recordings. Boezeman et al. describes repeating an audio 
segment 3.3 times to fill an entire 100 second length of animation. (See Boezeman et al. . 
Col. 7, lines 61-65 and Fig. 10.) The tempo is not adjusted to fit the audio recording to a 
section of the recorded signal marked by the reference as recited, for example, by Claim 21. 

Baumgartner et al. and Boezeman et al. fail to combine to disclose, suggest, or teach 
adjusting the tempo of at least a portion of an audio recording to "fit the first audio recording 
to a section of the second audio or video recording." Accordingly, Applicants respectfully 
request withdrawal of the rejection of Claim 21 and its dependent claims, Claims 22-26. 

C. The Examiner's Rejection of Claims 1-26 Should be Reversed Because 
There is No Suggestion to Combine the Teachings of Baumgartner et ah with 
Those of Boezeman et al. 

To establish a prima facie case of obviousness based on a combination of prior 
art references under 35 U.S.C. § 103(a), the Examiner must first show that there is a 
suggestion or motivation to combine the teachings of those references. This may come in the 
form of some objective teaching in the prior art or, alternatively, knowledge generally 
available to one of ordinary skill in the art at the time of the invention that would lead that 
individual to combine the relevant teachings of the references. 
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When the motivation to combine the teachings of the references is not 

immediately apparent, it is the duty of the Examiner to explain why the combination of the 

teachings is proper. Ex parte Skinner , 2 USPQ.2d 1788 (Bd. Pat. App. & Inter. 1986). In this 

case, the Examiner has not shown - and indeed, cannot show - that there would have been 

any motivation or suggestion to one of skill in the art to combine the teachings of Boezeman 

et al. and Baumgartner et al. In support of the obviousness rejection, the Examiner states: 

It would have been obvious to one having ordinary skill in the 
art at the time the invention was made to include Boezeman' s 
teachings of providing a tempo for an audio to be at least 
partially included in the recording signal, the tempo to fit the 
audio recording to a section of the recorded signal marked by 
the reference into that of Baumgartner' s adjusted tempo 
invention. By doing so, the system would be enhanced by 
providing a graphically editor to an end user wherein the user 
can graphically perform editing and synchronizing audio 
and video as well as animation tool based on user's desired 
taste. 

(Page 18 of the Office Action, dated November 9, 2004, boldface in original.) Appellants 

respectfully disagree. Neither reference provides a suggestion to combine the processing 

technique of Baumgartner et al. and the multi-media editor of Boezeman et al. 

Boezeman et al. describes a multimedia application that a human application 

developer might use to create a production "from parts on a scene that have been selected 

previously." (Col. 5, lines 40-41). Boezeman et al. states: 

The sequence editor is a mechanism that allows a user to 
synchronize relative time, absolute time and event time 
together in an easy to use manner. The solution assumes that 
items for synchronization (such as an animation, video, 
audio, image) have been previously identified and selected 
by a user . 

f Boezeman et al.. Col. 5, lines 47-52, Emphasis added). Boezeman et al. requires interaction 
with the user to operate. (See, for example, Boezeman et al. . Col. 7, lines 5-6, lines 11-13, 
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lines 23-25, lines 32-33, lines 41-43, and lines 52-53.) The user of the system described by 
Boezeman et al. selects saved multimedia files and builds a new production from them. 
Instead of relying on a developer to build a new production as described in Boezeman et al. , 
Baumgartner et al. describes a computer system that "synchronizes the audio and video data 
streams during a multimedia presentation to ensure that the appropriate sounds are generated 
by the speaker 132 when the corresponding images are being displayed by the video monitor 
122." (Baumgartner et al. . Col. 9, lines 9-13). 

As a result of their fundamental differences, Baumgartner et al. and Boezeman et al. 
cannot work together in combination. First, the principal of operation of one is completely 
different than the principal of operation of the other. Baumgartner et al. automates the 
synchronization of audio and video data streams during a presentation — avoiding the use of 
human interaction. Boezeman et al. enables users to create a production from existing 
multimedia files — enhancing the functionality available using human interaction. Second, a 
person of skill in the art would not combine a reference describing a production system used 
to create a presentation ( Boezeman et al. ) with a reference describing a "method for 
synchronizing the audio and video data streams during a multimedia presentation." 
( Baumgartner et al.. Col. 7, lines 25-27, emphasis added.) 

In conclusion, Appellants respectfully submit that there is no suggestion or motivation 
to combine the teachings of Baumgartner et al. and Boezeman et al. to obtain the subject 
matter recited in claims 1-26. Accordingly, it is respectfully submitted that claims 1-26 are 
patentable under 35 U.S.C. § 103. 
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CONCLUSION 



In view of the foregoing, the Appellants submit that: 

Claims 1-26 are not properly rejected under 35 U.S.C. § 103(a) over the combination 
of Baumgartner et al. „ and Boezeman et al . and are patentable. 

Accordingly, Appellants respectfully request that the Board reverse all claim 
rejections and indicate that a Notice of Allowance respecting all pending claims should be 
issued. 



Respectfully submitted, 



FOLEY & LARDNER LLP 

Customer Number: 23524 

Telephone: (608) 258-4292 
Facsimile: (608) 258-4258 



Date January 4, 2005 



By 




Paul S. Hunter 
Attorney for Applicant 
Registration No. 44,787 
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APPENDIX - THE CLAIMS ON APPEAL 

1 . A method of adjusting tempo of an audio recording to match audio events to 
video or other audio events in an audio-visual recording, the method comprising: 

receiving a reference indicating a location in a recorded signal, the reference 
being indicative of a desired audio tempo change location in the recorded signal; and 

providing a tempo for an audio recording to be at least partially included in the 
recorded signal, the tempo being adjusted to fit the audio recording to a section of the 
recorded signal marked by the reference. 

2. The method of claim 1, wherein the reference is indicative of a time location 
in the recorded signal to coincide a musical event with a particular frame of video in the 
recorded signal. 

3. The method of claim 1, wherein the reference is indicative of a location in the 
audio recording to be synchronized with a cursor time reference located in the recorded 
signal. 

4. The method of claim 1 , further comprising providing a user interface via a 
computing device, the user interface providing graphical representations of the recorded 
signal and of the audio recording to be at least partially included in the recorded signal. 

5. The method of claim 4, wherein the graphical representations include an audio 
waveform, wherein the user interface provides for the selective manipulation of 
characteristics of the audio waveform. 

6. The method of claim 5, wherein the selective manipulation provided by user 
interface includes providing for the increase in length of the audio waveform, thereby 
increasing the duration of the audio recording to be at least partially included in the recorded 
signal. 

7. The method of claim 1, wherein the step of providing a tempo for an audio 
recording to be at least partially included in the recorded signal comprises receiving a 
indication of a beginning and an end of the audio recording segment. 
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8. The method of claim 1 5 further comprising displaying video thumbnails of 
video images in the recorded signal on a user interface, the user interface having time 
indications labeling the video thumbnails according to timing of appearance of video images 
in the recorded signal. 

9. The method of claim 8, further comprising displaying audio representations of 
the audio recording to be at least partially included in the recorded signal, the audio 
representations being labeled with the time indications. 

10. In a computer program product, a system of determining the tempo of a 
portion of music such that one tempo phrase ends and another tempo phrase begins at a frame 

* of video or portion of audio as desired by a user of the computer program product, the system 
comprising: 

means for receiving a reference indicating a location in a recorded signal; and 

means for providing a tempo for an audio recording segment to be included in 
the recorded signal, the tempo being adjusted to fit the audio recording segment to a section 
of the recorded signal marked by the reference. 

1 1 . The system of claim 1 0, further comprising means for interfacing with a 
computing device, the interfacing means being configured to provide graphical 
representations of the recorded signal including video images and of the audio recording 
segment to be included in the recorded signal. 

12. The system of claim 10, wherein the means of providing a tempo for an audio 
recording segment to be included in the recorded signal comprises means for receiving a 
indication of a beginning and an end of the audio recording segment. 

13. The system of claim 10, further comprising means for displaying video 
thumbnails of video images in the recorded signal on a means for interfacing with a 
computing device, the interface means having time indications labeling the video thumbnails 
according to timing of appearance of video images in the video. 
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14. The system of claim 13, further comprising means for displaying audio 
representations of the audio recording segment to be included in the recorded signal, the 
audio representations being labeled with the time indications. 

15. A processing system comprising: 

a central processing unit (CPU); and 

a storage device coupled to a processor and having stored there information 
for configuring the CPU to: 

receive a reference indicating a location in a recorded signal; and 

provide a tempo for an audio recording segment to be included in the 
recorded signal, the tempo being adjusted to fit the audio recording segment to a section of 
the recorded signal marked by the reference. 

16. The system of claim 15, further comprising a presentation device, wherein the 
presentation device is configured to provide a graphical user interface which presents 
portions of the recorded signal and the audio recording segment. 

17. The system of claim 15, further comprising an interface device configured to 
connect the CPU with a network of computers. 

1 8. The system of claim 15, wherein the storage device having stored files 
containing video image information. 

19. The system of claim 1 5, wherein the CPU is further configured to assign the 
provided tempo to the audio recording segment. 

20. The system of claim 15, wherein the CPU is further configured to save a file to 
the storage device, the file including information related to the video, the audio recording 
segment, and the provided tempo. 

21. A graphical user interface configured to display representations of audio 
signals and video signals and being further configured to provide for creation of an audio or 
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an audio visual production using a plurality of audio or video recordings, the graphical user 
interface comprising: 

a first graphical display area on which graphical representations of a first 
audio recording can be displayed; 

a second graphical display area on which graphical representations of a second 
audio or video recording can be displayed; and 

a reference marker which is configured to be selectively located by a user, the 
reference marker being used to adjust the tempo of at least a portion of the first audio 
recording,_the tempo adjustment being provided to fit the first audio recording to a section of 
the second audio or video recording. 

22. The graphical user interface of claim 21, wherein the reference marker is a 
location marker indicating a measure location in the first audio recording. 

23. The graphical user interface of claim 22, wherein the tempo adjustment is 
performed using the reference marker in the first audio recording and a cursor position in the 
second audio or video recording. 

24. The graphical user interface of claim 21, wherein the reference marker is a 
time marker indicating a time location in the second audio or video recording. 

25. The graphical user interface of claim 24, wherein the tempo adjustment is 
performed using the reference marker in the second audio or video recording and a cursor 
position in the first audio recording. 

26. The graphical user interface of claim 24, wherein the tempo adjustment is 
performed using the reference marker in the second audio or video recording and a position 
in the first audio recording to which a user drags the reference marker. 
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